The Hidden 25 Percent: An Analysis of Nonresponse on the 1980 General Social Survey

Methods for estimating nonresponse bias are reviewed and several methods are tried on the 1980 GSS. The results indicate that various estimating procedures are inappropriate and that even the more promising techniques can provide faulty estimates of nonresponse bias. By its nature, nonresponse bias is very difficult to assess accurately and no simple, certain method exists. Tom W. Smith is Senior Study Director, National Opinion Research Center, University of Chicago. This research was done for the General Social Survey Project directed by James A. Davis. The project is supported by the National Science Foundation, Grant No. SOC77-03279. This is an abridged version of GSS Technical Report No. 25 published by NORC, 1981. The author wishes to thank James A. Davis, Howard Schuman, and Stanley Presser for their comments. Public Opinion Quarterly Vol. 47:386-404 ? 1983 by the Trustees of Columbia University Published by Elsevier Science Publishing Co., Inc. 0033-362X/83/0047-386/$2.50 NONRESPONSE ON THE 1980 GSS 387 ponse rate, we do not know the nonresponse mean since we have no measure of Y among nonrespondents. Two alternatives are usually presented in discussing nonresponse-how to minimize nonresponse and how to estimate and correct for differences between the respondents and nonrespondents. In this paper we ignore the first alternative, accepting that a nonresponse rate of .25 is typical for good, state-of-the-art surveys (Smith, 1978; Davis et al., 1980; and Groves and Kahn, 1979). Instead, we will review the various existing approaches to estimating the characteristics of nonrespondents and then apply several of the proposed approaches to nonresponse on the 1980 GSS. Measuring Nonrespondents and Assessing Nonresponse Bias Numerous methods have been proposed to estimate the attributes of nonrespondents (Daniel, 1975). Some are appropriate for certain types of surveys (e.g., list samples only) while others can be used with modification across various methods of administration with various sample frames (e.g., from mail lists to RDD telephone). Attention will focus primarily on methods that are appropriate, or at least have been offered as appropriate, for face-to-face, national surveys. Among other things, this eliminates list samples where information about the respondent is known prior to the survey. Our review of nonresponse studies found nine major approaches to assess and adjust for nonresponse: 1. External population checks 2. Geographic/aggregate level data 3. Interviewer estimates 4. Interviewing nonrespondents about nonresponse 5. Subsampling of nonrespondents 6. Substitution for nonrespondents 7. Politz-Simmons adjustment 8. Extrapolation based on difficulty 9. Conversion adjustments Probably the simplest check is to compare sample estimates (usually distributions) to some universe figures or preferred sample estimates such as the U.S. Census or the Current Population Survey (Crossley and Fink, 1951; Stephen and McCarthy, 1958: Smith, 1979; and Presser, 1981). Strictly speaking, when using such a criterion comparison, one is not checking how much difference comes from nonresponse but how much comes from nonresponse and all other